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Executive  Summary 


The  Weapon  Systems  Acquisition  Refonn  Act  of  2009  (WSARA)  requires  root 
cause  analyses  (RCA)  be  conducted  for  major  defense  acquisition  programs  (MDAPs)  as 
part  of  the  Nunn-McCurdy  breach  certification  process.  In  2010  and  2011,  the  Director, 
Perfonnance  Assessments  and  Root  Cause  Analyses  (PARCA)  conducted  fourteen  RCAs 
with  the  support  of  RCA  reports  by  the  Institute  for  Defense  Analyses  (IDA)  and  the 
RAND  Corporation.  The  RCAs  covered  fifteen  programs,1  eleven  MDAPs  that  had  a 
critical  Nunn-McCurdy  breach  and  one  MDAP  and  three  major  automated  infonnation 
systems  (MAISs)  that  had  severe  cost  growth  issues. 

Are  most  instances  of  extreme  cost  growth  in  MDAPs  caused  primarily  by  one  or 
another  of  a  few  root  causes — for  example,  incorporation  of  immature  technologies, 
unrealistic  cost  estimates,  inadequate  program  oversight?  PARCA  tasked  IDA  to 
investigate  this  question  by  examining  the  IDA  and  RAND  RCAs  it  had  sponsored  and 
published. 

Methodology 

The  IDA  and  RAND  RCAs  reported  their  findings  at  various  levels  of  detail  and  in 
different  tenns.  To  assess  the  degree  of  commonality  among  the  reported  root  causes, 
IDA  had  first  to  establish  a  taxonomy  that  would  record  and  classify  these  root  causes 
systematically. 

The  guidance  in  WSARA  suggests  eight  possible  root  causes  of  cost  growth: 

1 .  Unrealistic  perfonnance  expectations; 

2.  Unrealistic  baseline  estimates  for  cost  or  schedule; 

3.  Immature  technologies  or  excessive  manufacturing  or  integration  risk; 

4.  Unanticipated  design,  engineering,  manufacturing,  or  technology  integration 
issues  arising  during  program  perfonnance; 

5.  Changes  in  procurement  quantities; 

6.  Inadequate  program  funding  or  funding  instability; 


The  Advanced  Threat  Infrared  Countermeasures/Common  Missile  Warning  System  (ATIRCM/CMWS) 
RCA  reported  on  the  system’s  two  subprograms,  ATIRCM  and  CMWS,  separately. 
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7.  Poor  performance  by  government  or  contractor  personnel  responsible  for 
program  management;  or 

8.  Any  other  matters. 

The  RAND  and  IDA  RCAs  turned  out  to  be  much  more  complicated.  Numerous 
findings  were  placed  in  the  “Any  other  matters”  category,  and  some  causes  with  unique 
origins  landed  in  the  “Unrealistic  baseline  estimates”  or  “Poor  performance  by 
government  or  contractor  personnel”  categories;  thus,  the  WSARA  taxonomy  lacked 
sufficient  detail  to  capture  the  diversity  of  root  causes. 

In  practice,  PARCA  grouped  the  root  causes  into  three  major  categories — issues  at 
program  inception,  issues  during  program  execution,  and  exogenous  factors — in  their 
RCA  reports.  The  distribution  of  root  causes  before  and  during  program  execution 
measures  the  balance  of  “birth  defects”  to  “bad  parenting”  and  indicates  the  extent  to 
which  there  are  opportunities  for  remediation  within  troubled  programs. 

This  IDA  report  adopts  PARCA’s  top-tier  categories,  discussed  in  the  previous 
paragraph;  retains,  supplements,  and  modifies  many  of  the  issues  identified  one  level 
down  in  the  WSARA  taxonomy  as  the  “middle”  tier;  and  lists  the  detailed  root  causes  as 
even  lower  tiers  to  record  the  exact  findings  in  IDA  and  RAND  RCAs.  This  layout  allows 
IDA  to  identify  common  issues  among  these  RCAs.  IDA’s  three-tier  root  causes 
taxonomy  is  provided  in  Table  2  and  the  detailed  tabulation  of  the  RCA  findings  is  in 
Table  A-l  in  Appendix  A. 

Findings 

The  tables  and  figure  in  this  Executive  Summary  present  the  results  of  IDA’s 
assessment  of  the  significant  root  causes  of  cost  growth  as  reported  in  the  RAND  and 
IDA  RCA  reports  at  the  top  and  middle  tier  levels  only. 
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•  Program  has  at  least  one  mid-tier  issue  within  the  top  tier 
•/  Middle  tier  issue  reported  in  IDA  and  RAND  RCAs 


At  the  top  level  (see  the  Program  Count  by  Top  and  Middle  Tier  Issues  figure 
below),  root  causes  are  spread  relatively  evenly  between  issues  at  inception,  issues  during 
execution,  and  exogenous  factors.  At  the  middle  tier,  flawed  Milestone  (MS)  II"  cost 
estimates  (twelve  programs),  exogenous  directed  changes  (nine  programs),  and  lack  of 
acquisition  rigor  (eight  programs)  during  execution  stand  out  from  other  general  causes, 
with  each  of  those  containing  more  than  half  of  the  programs  in  the  sample.  IDA 
grouped  all  instances  where  MS  B  decisions  were  made  without  rigorous  deliberation  on 
contract  strategy,  requirements,  design  maturity,  acquisition  plan,  or  cost  estimate  into 
the  “lack  of  acquisition  rigor”  category.  Twelve  of  the  programs  surveyed  reported  at 
least  two  of  these  prominent  mid-tier  causes;  Global  Hawk,  CHEM-DEMIL  ACWA,  and 
Apache  LB-Blk  III  reported  all  three  causes.  Evolving  requirements  and  scope  (five 
programs),  inadequate  government  management  (six  programs)  and  unanticipated  design 
issue  (six  programs)  are  middle  tier  issues  cited  for  a  third  of  these  programs. 


Program  Count  by  Top  and  Middle  Tier  Issues 
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MS  II  was  the  decision  point  to  enter  into  Engineering  &  Manufacturing  Development  phase  for 
programs  initiated  before  2000.  DoDI  5000.2  issued  on  October  23,  2000  designated  MS  B  as  the 
entrance  point  into  System  Development  and  Demonstration  phase,  which  was  reversed  back  to 
Engineering  and  Manufacturing  Development  phase  in  DoDI  5000.2  of  December  2,  2008.  In  this 
report,  we  use  MS  B  and  MS  II  interchangeably. 

Table  A-l  in  Appendix  A  provides  a  more  detailed  version  of  the  above,  and  assigns  mid-  and  low-tier 
causes  to  the  fifteen  programs. 
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**  Average  Procurement  Unit  Cost  (APUC)  and  Program  Acquisition  Unit  Cost  (PAUC)  growths  measured 
against  program's  current  baseline 

The  number  of  specific  root  causes  per  program  varies  substantially  across  the 
sample  of  fifteen  programs.  Navy  Enterprise  Resource  Planning  (ERP)  has  no  issues, 
DDG  1000  and  Excalibur  have  two  issues  each,  and  Global  Hawk  has  eight  root  causes 
responsible  for  its  Nunn-McCurdy  breach.4  Interestingly,  the  number  of  root  causes  per 
program  does  not  appear  to  be  related  to  the  degree  of  cost  growth  for  that  program.  The 
two  programs  with  the  highest  unit  cost  growth — ATIRCM  and  Excalibur — have  four  or 
fewer  significant  root  causes  of  cost  growth  each,  and  the  program  with  the  greatest 
number  of  root  causes — Global  Hawk — has  the  smallest  Program  Acquisition  Unit  Cost 
(PAUC)  and  second  smallest  Average  Procurement  Unit  Cost  (APUC)  growth  of  those 
surveyed. 

Conclusion 

IDA  was  commissioned  to  identify  common  root  causes,  if  any,  of  cost  growth 
across  fifteen  programs  as  reported  in  the  RCAs  by  RAND  and  IDA.  These  programs 
experienced  extreme  cost  growth  and  represent  only  a  fraction  of  all  MDAPs. 
Observations  on  these  RCA  reports  should  not  be  generalized.  The  RCAs  reported  many 
different  root  causes  of  cost  growth  for  the  programs.  Some  of  the  apparent  differences 
could  be  attributed  to  the  individual  RCA  selection  process,  or  the  choice  of  phrases  to 
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See  Table  A-l  in  Appendix  A  for  the  assignment  of  lower-tier  root  causes  to  the  fifteen  programs. 


describe  the  root  cause.  We  expanded  the  root  causes  taxonomy  to  record  the  RCA 
findings  as  reported  with  no  adjustment  to  account  for  their  inherent  subjectivity. 
Notwithstanding  this  limitation,  five  specific  conclusions  can  be  drawn  from  this 
exercise.  First,  the  survey  shows  that  the  root  causes  of  a  program’s  cost  growth  are 
many.  The  reported  root  causes  vary  from  program  to  program.  Most  programs  have 
more  than  one  significant  root  cause.  These  observations  from  the  RCA  reports  stand  in 
contrast  to  “single  cause”  theories,  which  attribute  extreme  cost  growth  across  the  MDAP 
portfolio  primarily  to  a  single  factor  such  as  immature  technologies,  inadequate  systems 
engineering,  or  acquisition  “culture”  at  DoD.  Although  some  root  causes — like 
unrealistic  schedules,  unanticipated  design  issues,  quantity  reductions — were  common  to 
several  programs,  none  were  present  in  more  than  two-thirds  of  the  RCA  reports. 
Furthennore,  these  factors  were  never  the  sole  source  of  cost  growth  identified  for  the 
program  in  question.  It  is  clear  that  the  causes  of  extreme  cost  growth  are  many,  not  one. 

Second,  from  the  prevalence  of  causes  of  cost  growth  at  program  inception  in  the 
RCA  reports,  we  can  conclude  that  the  rigor  of  the  MS  B  process  is  important  for 
controlling  extreme  cost  growth.  Fourteen  of  the  fifteen  RCA  programs  identified  issues 
at  inception  as  causes  of  cost  growth.  Unrealistic  cost  and  schedule  estimates,  and  flawed 
contract  and  contract  incentive  structures  have  been  cited  in  several  RCAs.  Once 
embedded,  issues  at  inception  are  difficult  to  overcome.  A  more  rigorous  and  accountable 
MS  B  process  would  help  to  limit  the  incidence  of  “birth  defects”  that  lead  to  extreme 
cost  growth  in  MDAPs. 

Third,  exogenous  factors  often  contribute  to  cases  of  extreme  cost  growth.  Nearly 
two-thirds  of  the  RCA  reports  (nine  of  fifteen  programs)  cited  exogenous  quantity 
reductions  or  requirements  changes  as  responsible  for  cost  growth.  Directed  quantity 
reduction  drives  up  PAUC,  and  requirements  changes  and  scope  expansions  lead  to 
numerous  contract  modifications  and  increase  the  program  acquisition  cost.  To  the  extent 
that  these  factors  are  truly  exogenous  to  program  initiation  and  execution,  their  influence 
on  cost  growth  has  the  effect  of  exaggerating  the  severity  of  MDAP  perfonnance 
shortcomings  in  the  data.  Measures  to  control  requirements  growth  and  acknowledge  the 
costs  of  directed  changes  could  be  implemented  at  program  restructure  to  manage  the 
influence  of  such  exogenous  factors. 

Fourth,  the  RCA  reports  suggest  that  proper  management  of  MDAPs  is  critical  for 
controlling  extreme  cost  growth.  Roughly  half  (eight  of  fifteen  programs)  of  the  RCAs 
reported  inadequate  government  oversight  or  ineffective  contract  management  as 
significant  causes  of  cost  growth.  Causes  at  program  inception  and  exogenous  factors  can 
be  unambiguously  associated  with  given  amounts  of  cost  growth,  and  poor  management 
and  oversight  issues  act  to  exacerbate  the  consequences  of  cost  growth,  and  the  tie  is 
difficult  to  quantify.  Nevertheless,  these  execution  issues  are  more  important  to  be 
identified,  because  they  are  areas  for  improvement  and  remedial  actions. 
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Finally,  this  exercise  highlights  the  need  for  improved  root  cause  taxonomy  beyond 
that  already  suggested  by  WSARA  and  PARCA.  Although  the  IDA  research  team 
proposed  an  alternative  taxonomy,  it  did  so  by  allowing  the  RCA  reports,  which  used 
WSARA  as  a  guide,  to  largely  detennine  the  structure  beneath  PARCA’s  top  tier 
allocation  of  issues  at  inception,  issues  during  execution,  and  exogenous  factors.  To 
improve  the  consistency,  understanding,  and  usefulness  of  cross-program  RCAs,  more 
rigor  is  needed  to  structure  future  root  cause  taxonomies.  A  broader  selection  of  programs 
and  participants  would  serve  well  in  future  efforts  towards  this  end.  Along  with 
improving  the  rigor  of  the  MS  B  process,  untangling  the  role  of  management 
perfonnance,  and  acknowledging  the  influence  of  exogenous  factors,  the  development  of 
a  cross-program  RCA  structure  stands  as  a  key  opportunity  for  improvement. 
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A.  Introduction 


Between  2010  and  2011,  the  Director,  Perfonnance  Assessments  and  Root  Cause 
Analyses  (PARCA)  performed  fourteen  root  cause  analyses  (RCA)  for  fifteen  programs1 
that  had  severe  cost  growth  issues.  Pursuant  to  conducting  these  RCAs,  PARCA 
commissioned  independent  reports  from  the  Institute  for  Defense  Analyses  (IDA)  and  the 
RAND  Corporation,  examining  the  root  causes  of  these  programs’  cost  growth. 

Are  most  instances  of  extreme  cost  growth  in  major  defense  acquisition  programs 
(MDAPs)  caused  primarily  by  one  or  another  of  a  few  root  causes — for  example, 
incorporation  of  immature  technologies,  unrealistic  cost  estimates,  and  inadequate 
program  oversight?  PARCA  tasked  IDA  to  investigate  this  question  by  examining  the 
IDA  and  RAND  RCAs  that  had  been  sponsored  and  published. 

B.  Identification  of  the  Programs  Examined 

This  report  covers  the  results  of  eight  IDA  and  seven  RAND  RCAs  (Table  1).  The 
list  contains  fifteen  reports  for  PARCA’s  fourteen  RCAs,  because  IDA  and  RAND  were 
both  asked  to  perfonn  an  RCA  on  the  Joint  Strike  Fighter  program,  F-35,  independent  of 
each  other.  The  programs  included  in  this  report  consist  of  ten  MDAPs  that  experienced  a 
critical  Nunn-McCurdy  breach  between  2010  and  2011  and  one  MDAP  and  three  major 
automated  infonnation  systems  (MAISs)  that  had  significant  cost  growth  issues.  One 
MDAP  RCA  (ATIRCM/CMWS  (Advanced  Threat  Infrared  Counter  Measure/Common 
Missile  Warning  System))  consisted  of  two  subprograms;  therefore,  this  analysis 
compares  the  root  causes  for  cost  growth  on  fifteen  programs  in  total. 

The  programs  in  this  report  are  a  small  fraction  of  the  Department  of  Defense 
(DoD)’s  acquisition  portfolio;  they  are,  however,  a  reasonable  cross-section  of  the 
programs.  The  programs  are  distributed  evenly  across  the  Services  with  three  programs 
each  from  the  Anny  and  the  Navy,  four  from  the  Air  Force,  one  from  the  Marine  Corps, 
and  three  joint  programs.  Furthermore,  they  present  a  variety  of  system  types,  including 
ships,  satellites,  tactical  fighters,  electronics,  and  MAISs.  This  diversity  of  programs 
affords  IDA  the  opportunity  to  assess  the  degree  of  commonality  in  factors  contributing 
to  critical  cost  growth  across  a  wide  spectrum  of  defense  programs.  However,  due  to  the 
fact  that  each  program  has  experienced  or  nearly  experienced  a  critical  Nunn-McCurdy 
breach,  lessons  drawn  from  the  RCAs  may  only  be  applicable  to  other  cases  of  extreme 
cost  growth  and  may  not  be  generalized  to  the  broader  MDAP  portfolio,  in  which  similar 
cost  growth  is  not  always  present. 


The  ATIRCM/CMWS  RCA  reported  two  subprograms,  ATIRCM  and  CMWS,  separately.  Hence  these 
fourteen  PARCA  RCAs  were  performed  for  fifteen  programs. 
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Table  1.  PARCA  RCAs 


Program 

Analysis 

Source 

GCSS-MC* 

Global  Combat  Support  System  -  Marine  Corps:  Root  Cause 
Analysis  (2012) 

IDA 

ECSS* 

Expeditionary  Combat  Support  System:  Root  Cause  Analysis 
(2011) 

IDA 

Navy  ERP* 

Root  Cause  Analysis  Serial  #2:  Navy  ERP,  Excalibur,  Root 

Cause  Analyses  Methodology  and  Data  Sources  (2011) 

RAND 

CHEM-DEMIL 

ACWA 

Chemical  Demilitarization  -  Advanced  Chemical  Weapons 
Alternatives:  Root  Cause  Analysis  (2011) 

IDA 

ATIRCM-CMWS 

Root  Cause  Analysis  for  the  ATIRCM/CMWS  Program  (2010) 

IDA 

Global  Hawk 

Global  Hawk:  Root  Cause  Analysis  of  Projected  Unit  Cost 

Growth  (2011) 

IDA 

F-35 

WSARA  2009:  Joint  Strike  Fighter  Root  Cause  Analysis  (2010); 
Root  Cause  Analysis  of  Nunn-McCurdy  Breaches,  Volume  1: 
Zumwalt  Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache 
and  Wideband  Global  Satellite  (2011) 

IDA; 

RAND 

RMS 

Remote  Minehunting  System:  Root  Cause  Analysis  (2010) 

IDA 

FAB-T 

Root  Cause  Analysis  of  Family  of  Beyond  Line-of-Sight 
Terminals(2012) 

IDA 

Apache  Block  III 

Root  Cause  Analysis  of  Nunn-McCurdy  Breaches,  Volume  1: 
Zumwalt  Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache 
and  Wideband  Global  Satellite  (2011) 

RAND 

DDG  1000 

Root  Cause  Analysis  of  Nunn-McCurdy  Breaches,  Volume  1: 
Zumwalt  Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache 
and  Wideband  Global  Satellite  (2011) 

RAND 

WGS 

Root  Cause  Analysis  of  Nunn-McCurdy  Breaches,  Volume  1: 
Zumwalt  Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache 
and  Wideband  Global  Satellite  (2011) 

RAND 

Excalibur 

Root  Cause  Analysis  Serial  #2:  Navy  ERP,  Excalibur,  Root 

Cause  Analyses  Methodology  and  Data  Sources  (2011) 

RAND 

JTRS  GMR 

Root  Cause  Analysis:  Joint  Tactical  Radio  Systems  Ground 

Mobile  Radar  (2011) 

RAND 

‘Growth  as  reported  by  GAO.  PAUC/APUC  growth  not  applicable  for  ERP  acquisitions. 


C.  Methodology 

To  assess  the  degree  of  commonality  among  root  causes  of  cost  growth,  IDA  first 
reviewed  these  fifteen  RCA  reports,  performed  by  IDA  and  RAND,  and  then  selected  an 
appropriate  taxonomy  for  these  RCAs  in  order  to  document  and  score  the  findings 
systematically.  IDA  chose  to  use  the  taxonomy  in  Table  2,  instead  of  the  root  cause 
classification  listed  in  the  Weapon  Systems  Acquisition  Refonn  Act  (WSARA),  to 
replicate  the  exact  findings  reported  in  the  IDA  and  RAND  RCAs. 
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Table  2.  IDA  Root  Causes  Taxonomy 

1.0  Issues  at  Inception 

1.1  Unrealistic  Performance  Expectation 

1.2  Immature  Technology 

1 .3  Lack  of  Acquisition  Rigor3 

1.3.1  Flawed  Contracting  Strategy/Structure/Incentive 

1.3.2  Flawed  Engineering  Strategy 

1.3.3  Immature  Design 

1 .3.4  Adopted  Unrealistic  Cost  Estimate 

1.3.5  Ambitious  Requirement 

1.4  Flawed  MS  II  Estimate 

1.4.1  Unrealistic  Schedule 

1 .4.2  Unrecognized  Technology  Requirement/Complexity 

1 .4.3  Erroneous  Cost  Estimate 

1.4.4  Unrealistic  Cost  Estimate 

1.4.5  Flawed  Weight  Estimate 

1.4.6  Flawed  Programmatic  Assumptions 

1.4.7  Inadequate  Risk  Allowances 
2.0  Issues  during  Execution 

2.1  Unanticipated  Design  Issues 

2.2  Fragmented  and  Diffused  Management  Structure 

2.3  Evolving  Requirements  and  Scope 

2.4  Poor  Contractor  Performance 

2.4.1  Scheduled  Delivery/Defect  Rate/Reliability 

2.4.2  Poor  Management  of  Subcontractor 

2.5  Inadequate  Government  Management 

2.5.1  Inadequate  Oversight/Situation  Awareness 

2.5.2  Deficient  Management/Funding  Practice 

2.5.3  Poor  Schedule  Management 

2.5.4  Ineffective  Management  of  Contractor  Incentives 
3.0  Exogenous  Factors 

3.1  Directed  Changes/Priorities/Quantities 

3.2  Constricted  Environment  to  Control  Costb 

3.3  Shrinking  of  Business  Base/Emerging  of  Low  Cost  Competitor 

_ 3 A _ Inadequate  Program  Funding _ 

a  IDA  grouped  all  instances  where  Milestone  B  decision  was  made  without  rigorous  deliberation  on 

contract  strategy,  requirements,  design  maturity,  acquisition  plan,  or  cost  estimate  into  the  category 
“lack  of  acquisition  rigor.” 

b  Regulations,  international  treaties  and  other  considerations  that  constrict  cost  control  measures. 

WSARA  postulated  that  the  root  causes  of  cost  growth  could  be  attributed  to  one  of 
the  following  factors: 

1 .  Unrealistic  perfonnance  expectations; 

2.  Unrealistic  baseline  estimates  for  cost  or  schedule; 
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3.  Immature  technologies  or  excessive  manufacturing  or  integration  risk; 

4.  Unanticipated  design,  engineering,  manufacturing,  or  technology  integration 
issues  arising  during  program  performance; 

5.  Changes  in  procurement  quantities; 

6.  Inadequate  program  funding  or  funding  instability; 

7.  Poor  perfonnance  by  government  or  contractor  personnel  responsible  for 
program  management;  or 

8.  Any  other  matters. 

While  the  WSARA  taxonomy  has  been  recommended  to  guide  RCAs,  the  RAND 
and  IDA  RCA  research  presented  a  more  complex  picture  of  the  factors  contributing  to 
cost  growth.  Numerous  findings  were  placed  in  the  “Any  other  matters”  category,  and 
some  causes  with  unique  origins  landed  in  the  “Unrealistic  baseline  estimates”  or  “Poor 
perfonnance  by  government  or  contractor  personnel”  categories;  thus,  the  WSARA 
taxonomy  lacked  sufficient  detail  to  capture  the  diversity  of  root  causes.  Without  a 
conceptual  backbone,  the  WSARA  taxonomy  has  somewhat  limited  utility  to  the 
government  in  understanding,  remedying,  or  preventing  recurrent  cost  growth. 

In  practice,  PARC  A  has  followed  a  more  structured  format  in  their  RCA  reports. 
They  grouped  the  root  causes  into  three  major  categories:  issues  at  program  inception, 
issues  during  program  execution,  and  exogenous  factors.  PARCA  then  reported  root 
causes  in  greater  detail  within  each  group. 

This  IDA  report  adopts  PARCA’s  top-tier  categories,  discussed  in  the  previous 
paragraph;  retains,  supplements,  and  modifies  many  of  the  issues  identified  one  level 
down  in  the  WSARA  taxonomy  as  the  “middle”  tier;  and  extends  the  structure  further  by 
adding  lower-tier  issues  to  replicate  the  exact  findings  in  the  IDA  and  RAND  RCAs. 
Related  lower-tier  causes  are  then  grouped  under  an  overarching  mid-tier  cause.  For 
example,  WSARA ’s  root  cause  “Unrealistic  baseline  estimates  for  cost  or  schedule”  is 
modified  to  the  mid-tier  cause  “Flawed  Milestone  (MS)  II  Cost  Estimate,”  which 
includes  “Unrealistic  Schedule,”  “Erroneous  Cost  Estimate,”  “Unrealistic  Cost  Estimate,” 
and  other  root  causes  identified  by  IDA  and  RAND.  Mid-tier  causes  without  any  lower- 
tier  causes  are  themselves  root  causes.  From  the  IDA  and  RAND  RCAs  in  Table  1,  IDA 
derived  the  structured  taxonomy  on  root  causes  to  classify  RCA  findings  in  Table  2. 

Issues  at  program  inception  can  be  thought  of  as  a  program’s  “birth  defects”  and 
include  such  factors  as  unrealistic  cost  or  schedule  estimates,  immature  technology, 
unrealistic  performance  expectations,  poorly  defined  requirements  or  scope,  and  lack  of 
acquisition  rigor.  These  are  sources  of  cost  growth  inherent  in  the  Acquisition  Program 
Baseline  (APB).  Although  the  consequences  of  these  birth  defects  may  not  emerge  until 
program  execution,  DoD  managers  have  limited  leverage  to  remedy  issues  at  inception 
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once  incorporated  into  a  program’s  baseline  or  acquisition  strategy.  The  recognition  of 
common  issues  at  inception  is  valuable  to  guide  the  post-Nunn-McCurdy  restructuring 
process  and  to  apply  lessons  learned  from  flawed  program  initiations  to  new  acquisitions. 

Issues  during  program  execution  can  be  thought  of  as  “bad  parenting”  during  the 
development  (or  production)  phase  of  a  program.  Issues  during  execution  include 
unanticipated  design  difficulties,  poorly  functioning  management  structures,  inadequate 
government  management,  and  poor  contractor  performance  after  program  initiation.  By 
definition,  these  problems  cannot  be  traced  back  to  flaws  in  the  original  program  plans  or 
estimates,  and  they  signal  management  or  performance  shortcomings  above  and  beyond 
those  typical  for  similar  programs.  Unlike  issues  at  program  inception,  execution 
difficulties  may  be  remedied  midstream.  Heightened  oversight,  streamlined  management 
structures,  or  more  effective  contract  incentives,  for  example,  have  the  potential  to 
stabilize  cost  growth  for  a  program  suffering  from  execution  issues  following  (or  in 
anticipation  of)  a  Nunn-McCurdy  breach.  Knowledge  of  common  issues  during  execution 
affords  DoD  and  PARCA  opportunities  for  intervention  in  troubled  programs  to  control 
cost  growth. 

Directed  changes  in  quantity  or  requirements;  changes  in  political  or  funding 
priorities,  regulations,  treaty  requirements,  and  others  that  constrict  cost  control 
measures;  or  shifts  in  the  industrial  base  constitute  a  third  category  of  exogenous  factors 
contributing  to  cost  growth.  To  the  extent  that  these  factors  are  truly  external  to  the 
program — quantity  reductions  prompted  by  cost  overruns  and  scope  additions  correcting 
incomplete  requirements  do  not  meet  this  standard — exogenous  factors  and  their  resultant 
cost  overruns  must  be  considered  unavoidable.  The  three  basic  categories  in  IDA’s 
taxonomy — issues  at  inception,  issues  during  execution,  and  exogenous  factors — and 
their  constituent  causal  variants  therein  identify  the  character  and  locus  of  root  cause 
within  a  program’s  life  cycle  and  management  chain.  These  higher  levels  of  the 
taxonomy  are  roughly  consistent  with  PARCA’s  current  classification  structure,  which 
facilitates  greater  understanding,  remediation,  and  prevention  of  common  causes  of  cost 
growth  than  the  original  WSARA  taxonomy.  When  combined,  the  PARCA  classification 
structure  and  WSARA  taxonomy  compose  the  first  two  tiers  of  the  proposed 
classification  structure.  When  assessing  the  incidence  of  root  causes  across  programs, 
these  top  two  tiers  constitute  the  primary  level  at  which  common  causes  should  be 
measured. 

Beneath  the  top  two  tiers  of  the  taxonomy  lies  a  more  detailed  classification  of  the 
specific  causes  of  cost  growth  for  the  fifteen  programs  that  IDA  examined.  These  sub¬ 
categories  in  the  third  tier,  which  are  listed  in  Table  2,  correspond  to  root  causes  of  cost 
growth  documented  in  the  IDA  and  RAND  RCA  reports.  Because  the  survey  covers  a 
diverse  selection  of  programs,  the  makeup  of  this  lower  tier  was  detennined  primarily  by 
the  reports  themselves  in  order  to  give  a  more  detailed  perspective  rooted  in  the  unique 
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program  cost  growth  histories.  This  process  explains  why  many  of  the  sub-categories 
apply  to  only  one  program.  Accordingly,  causal  commonality  is  less  likely  to  arise  from 
the  lower  tier,  but  when  it  does,  commonality  may  be  indicative  of  specific,  recurring 
problems  across  programs  that  deserve  heightened  scrutiny. 

Many  of  the  programs  in  the  survey  had  multiple  causes  of  cost  growth.  In  such 
cases,  the  set  of  causes  was  first  narrowed  to  those  factors  judged  to  be  significant 
contributors  to  cost  overruns.  Even  where  the  RCAs  quantitatively  apportioned  cost 

growth  among  several  significant  causes,  no  attempt  was  made  to  discriminate  more 
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finely  among  those  significant  root  causes. 

The  individual  RCA  reports  reported  root  causes  at  various  levels  of  detail  and  in 
different  terms.  For  example,  one  report  reported  poor  government  performance  as  the 
root  cause,  while  another  elaborated  on  the  lack  of  oversight,  ineffective  management  of 
contractor  incentives,  and  poor  schedule  management.  Some  reports  identified  an 
unrealistic  cost  or  schedule  estimate  as  a  root  cause,  while  others  pointed  to  unrealistic 
schedule,  erroneous  estimate,  or  flawed  weight  estimate  as  the  issues.  At  the  lowest  level, 
RCA-identified  causes  detennined,  with  a  minor  degree  of  standardization  on 
terminology,  the  sub-categories  of  cost  growth.  This  allowed  IDA  to  align  a  reported  root 
cause  to  a  sub-category  closest  to  its  description.  Related  sub-categories  were  then 
consolidated  into  mid-level  issues  in  the  proposed  causal  taxonomy,  and  then  into  the 
three  top-tier  groups:  issues  at  inception,  issues  during  execution,  and  exogenous  factors. 

At  times,  the  results  might  differ  from  what  might  have  been  expected  from  the 
WSARA  taxonomy.  For  example,  the  ATIRCM  B-Kit  had  a  stringent  weight  requirement 
to  allow  its  installation  on  almost  all  Anny  helicopters.  The  procurement  quantity  cut 
became  inevitable  as  the  design  weight  exceeded  the  requirement,  leaving  CH-47  as  the 
only  helicopter  capable  of  carrying  it.  Hence,  the  ATIRCM  B-Kit  cuts  were  not 
exogenous,  and  the  root  cause  was  judged  to  be  an  unrealistic  performance  expectation  at 
program  inception.  Similarly,  when  the  Apache  Block  III  program  rejected  a  higher  DoD 
estimate  in  favor  of  a  lower  service  estimate  at  MS  B,  the  acquisition  decision  maker 
adopted  an  unrealistic  estimate,  a  case  for  lack  of  acquisition  rigor,  not  an  unrealistic  cost 
estimate. 

Upon  completion  of  the  alignment  process,  IDA  assessed  the  results  for 
commonality  in  root  cause  across  the  programs  in  the  survey,  where  commonality  of  root 
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For  example,  the  RAND  RCA  for  the  WGS  program  identified  and  apportioned  cost  growth  among 
three  independent  root  causes:  (1)  the  additional  four  Block  II  and  Ilf  satellites  (29  percent  of  cost 
growth);  (2)  movement  of  the  commercial  industrial  base  from  obsolete  WGS-compatible  technology 
(26  percent);  and  (3)  an  estimating  error  wherein  Block  I  and  II  satellites  were  budgeted  at  their  targeted 
(rather  than  ceiling)  contract  price  while  additional  Block  Ilf  satellites  were  budgeted  at  their  ceiling 
(not  targeted)  contract  prices,  resulting  in  an  artificial  15  percent  growth  in  unit  costs  (30  percent).  In 
IDA’s  evaluation  of  root  cause,  all  three  rose  to  the  level  of  significance  and  all  three  were  aligned  with 
the  proposed  taxonomy. 
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cause  was  assessed  primarily  at  the  top  two  levels.  Due  to  the  diversity  of  the  programs, 
specific  causal  subcategories  are  likely  to  vary  substantially  from  program  to  program. 
Where  commonality  is  present  at  the  lower  level,  it  may  indicate  a  specific  recurrent 
failure  that  deserves  further  investigation.  Moving  up  a  level,  commonality  among  the 
middle  tier  causes  may  indicate  general  shortcomings  in  the  acquisition  process,  such  as 
requirements  detennination  and  acquisition  strategy,  cost  and  schedule  estimation,  or 
program  and  contract  management.  Finally,  the  distribution  of  root  causes  before  and 
during  program  execution  measures  the  balance  between  birth  defects  and  bad  parenting 
and  indicates  the  extent  to  which  there  are  opportunities  for  remediation  within  troubled 
programs. 

D.  Limitations 

In  Appendix  A,  IDA  compiled  Table  A-l  to  record  the  root  causes  reported  in  the 
IDA  and  RAND  RCAs  on  fifteen  programs  at  the  lowest  tiers.  The  practice  of  assigning 
root  cause  based  solely  on  the  RCAs  allows  IDA  to  remain  an  objective  observer  but 
imposes  on  IDA’s  analysis  the  same  limitations  that  are  present  in  the  RCAs.  Three 
prominent  shortcomings  in  the  collection  of  RCAs  may  limit  the  generalizability  of  the 
results  to  the  broader  MDAP  portfolio:  (1)  the  stock  of  RCA  programs  is  small,  and  the 
programs  are  not  completely  representative  of  the  population  of  MDAPs  and  MAIS 
programs;  (2)  some  of  the  root  causes  are  interrelated  and  not  independent;  and  (3)  the 
root  causes  selection  process  varies  across  each  of  the  RCAs.  Due  to  these  limitations,  the 
table  should  be  used  only  for  observation,  not  for  generalization. 

The  survey  covers  fifteen  programs  with  RCA  reports:  GCSS-MC,  ECSS,  and  Navy 
ERP  are  MAIS  programs,  while  the  other  twelve  are  MDAP  programs.  Currently  there 
are  forty-one  active  MAIS  programs  and,  eighty-one  active  MDAPs.  Consequently,  these 
fifteen  programs  represent  only  a  fraction  of  MAISs  and  MDAPs.  Furthermore,  all  of  the 
programs  included  in  this  report  either  experienced  or  anticipate  a  critical  Nunn- 
McCurdy  breach.  The  majority  of  MDAPs  and  MAIS  programs  outside  of  this  report  will 
not  experience  such  extreme  cost  growth.  The  results  presented  here  should  not  be 
extended  to  programs  with  more  “normal”  cost  growth. 

The  survey  table  displays  the  root  causes  that  are  ostensibly  independent  of  each 
other.  This  layout  does  not  account  for  the  complex  interrelationships  and  interactions 
among  reported  root  causes.  For  example,  an  unrealistically  low  MS  B  cost  estimate 
could  be  caused  by  an  unrealistic  schedule  or  bv  unrecognized  technology  requirements 
and  complexity.  An  unrealistic  schedule  could  result  from  a  combination  of  optimistic 
design  maturity  assumptions,  ambitious  requirements,  inadequate  risk  allowance,  or  a 
compressed  schedule  by  direction.  Ineffective  management  of  contractor  incentives  could 
be  the  result  of  numerous  contract  modifications  to  accommodate  changes  in 
requirements  and  scope.  Depending  on  where  the  analysis  stops  along  the  causal  chain  of 
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cost  growth,  a  root  cause  identified  by  one  analyst  may  very  well  be  judged  as  only  a 
proximate  cause  in  passage  by  another  evaluator.  The  ultimate  entry  of  a  root  cause  at  the 
subcategory  level  of  the  above  taxonomy  represents  the  individual  RCA  choice,  with 
little  consistency  or  unifonnity.  Even  at  the  higher  level,  reported  root  causes  are  not 
completely  independent.  Cost  growth  that  generates  affordability  concerns  could  induce 
cuts  in  procurement  quantities  and  further  growth  in  unit  cost  (and  possibly  more  quantity 
reductions).  Unanticipated  design  issues  during  execution  could  have  their  origin  in 
immature  technology  at  the  program  inception.  Root  causes  of  cost  growth  are  so  innately 
intertwined,  at  this  point,  we  do  not  have  a  way  to  construct  an  RCA  taxonomy  totally 
free  of  interdependency  among  the  entries.  This  paper  made  no  attempt  to  dive  further 
into  causes  than  what  has  been  reported  in  the  RCAs  so  as  to  remain  objective  in  the 
survey. 

Finally,  the  ability  to  generalize  from  our  results  is  limited  by  variations  in  the  root 
cause  selection  process  across  the  RCA  reports.  The  RCAs  were  performed  by  analyst 
teams  in  IDA  and  RAND.  Each  team  followed  PARCA’s  general  guidance  to  collect 
program  histories,  locate  proximate  causes  of  cost  growth,  and  determine  the  root  causes 
of  cost  growth.  The  RCAs,  however,  are  noticeably  different  in  the  breadth  and  depth  of 
their  analyses  and  in  their  selection  of  root  causes.  The  differences  are  not  just  between 
RAND  and  IDA,  but  among  RCAs  within  RAND  and  IDA  perfonned  by  separate  teams 
of  analysts. 

The  DDG  1000,  WGS,  and  Excalibur  RCAs  indicated  one  or  two  root  causes — 
quantity  reduction  and/or  unrealistic  cost  estimate.  The  Global  Hawk  RCA  identified 
many — unexecutable  schedule,  missing  program  content,  and  unrealistic  cost  estimate 
were  defects  at  the  inception  of  the  program;  failures  of  contractor  execution,  program 
management,  and  Air  Force  oversight  and  funding  approach  are  shortcomings  during 
program  execution.  The  Remote  Minehunting  System  RCA  found,  in  addition  to  the 
commonly  cited  quantity  reduction  and  unrealistic  baseline  estimate  causes,  the  failure  of 
government  oversight  manifested  in  the  system’s  reliability  issues  to  be  a  significant  root 
cause  of  program  cost  growth. 

Both  IDA  and  RAND  were  commissioned  to  conduct  RCAs  on  the  F-35.  The 
research  reached  similar  conclusions,  but  the  rationales  behind  that  conclusion  were  quite 
different.  IDA  attributed  the  cost  growth  to  the  unrealistic  MS  B  cost  estimate  associated 
with  the  underestimation  of  airframe  unit  weight  and  other  errors.  RAND  found  that  the 
root  cause  lay  in  an  overly  optimistic  baseline  estimate  influenced  by  acquisition  reform 
and  produceability  initiatives.  Both  reports  identified  redesign  as  a  major  cause  of  F-35 
cost  growth,  but  could  not  offer  any  clear  or  compelling  account  of  its  root  causes.  IDA 
suggested  that  the  initial  low  airframe  weight  estimate  was  an  issue.  RAND  associated 
the  extra  cost  for  redesign  to  the  engineering  strategy  of  prototyping  Conventional  Take¬ 
off  and  Landing  (CTOL)  first  rather  than  Short  Take-off  and  Vertical  Landing  (STOVL) 
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during  system  design  and  development.  The  fact  that  two  independent  reports  reached 
somewhat  divergent  accounts  of  the  causes  of  cost  growth  for  a  single  program  casts 
doubt  on  the  consistency  of  the  root  cause  selection  process  across  different  programs 
and  reports. 

E.  Results 

Table  3,  Table  4,  and  Figure  1  through  Figure  4  present  the  results  of  IDA’s 
assessment  of  the  significant  root  causes  of  cost  growth  across  the  IDA  and  RAND  RCA 
reports  (Table  1)  done  for  PARC  A. 

At  the  top  level  (Table  3  and  Figure  1),  root  causes  are  spread  relatively  evenly 
between  issues  at  inception,  issues  during  execution,  and  exogenous  factors,  with  each 
category  contributing  roughly  30  percent  of  the  total  causal  count.  Beneath  the  top  level, 
flawed  MS  II  cost  estimates  (twelve  programs),  exogenous  directed  changes  (nine 
programs),  and  lack  of  acquisition  rigor  (eight  programs)  during  execution  stand  out  from 
other  general  causes,  with  each  category  represented  in  more  than  half  of  the  programs  in 
the  sample.  IDA  grouped  all  instances  where  MS  B  decisions  were  made  without 
rigorous  deliberation  on  contract  strategy,  requirements,  design  maturity,  acquisition 
plan,  or  cost  estimate  into  the  “lack  of  acquisition  rigor”  category.  Twelve  of  the 
programs  surveyed  reported  at  least  two  of  these  prominent  mid-tier  causes.  Global 
Hawk,  CHEM  DEMIL-ACWA,  and  Apache  LB-Blk3  reported  all  three  causes.  Evolving 
requirements  and  scope  (five  programs),  inadequate  government  management  (six 
programs)  and  unanticipated  design  issue  (six  programs)  were  middle  tier  issues  cited  for 
a  third  of  these  programs.  At  the  most  specific  level  (Table  A-l  and  Figure  2),  unrealistic 
development  and  testing  schedule  (six  programs)  and  unrecognized  technology 
requirement/complexity  (six  programs)  are  the  most  common  causes. 


Table  A-l  in  Appendix  A  provides  a  more  detailed  version  of  Table  3.  This  table  details  assignment  of 
mid-  and  low-tier  causes  to  the  fifteen  programs  and  provides  the  basis  for  the  results  presented  in 
Figure  1  through  Figure  4. 
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Table  3.  Root  Cause  Analyses  Summary 


Causes  of  Cost  and  Schedule 

Growth 

GCSS- 

MC 

ECSS 

Navy 

ERP 

ACWA 

ATIRCM 

CMWS 

Global 

Hawk 

F35  (JSF) 

RMS 

I- 

■ 

m 

< 

Li. 

Apache 

LB-Bklll 

DDG 

1000 

WGS 

Excalibur 

JTRS 

GMR 

Programs 

w/lssues 

Issues  at  Inception 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

14 

Unrealistic  Performance 
Expectation 

✓ 

s 

✓ 

3 

Immature  Technology 

✓ 

V 

✓ 

V 

4 

Lack  of  Acquisition  Rigor 

V 

V 

V 

V 

V 

V 

V 

8 

Flawed  Milestone  II  Cost  Estimate 

V 

V 

V 

✓ 

V 

V 

V 

V 

✓ 

V 

12 

Issues  During  Execution 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

11 

Unanticipated  Design  Issues 

✓ 

✓ 

s 

✓ 

6 

Fragmented  and  Diffused 
Management  Structure 

V 

2 

Evolving  Requirements  and  Scope 

V 

✓ 

s 

V 

✓ 

5 

Poor  Contractor  Performance 

V 

s 

V 

3 

Inadequate  Government 
Management 

V 

✓ 

✓ 

V 

6 

Exogenous  Factors 

• 

• 

• 

• 

• 

• 

• 

• 

• 

9 

Directed  Changes/Priorities/ 
Quantities 

✓ 

V 

✓ 

✓ 

✓ 

✓ 

V 

V 

9 

Constricted  Environment  to 

Control  Cost 

1 

Shrinking  Business  Base/ 

Emerging  Competitor 

V 

2 

Inadequate  Program  Funding 

V 

1 

•  Program  has  at  least  one  mid-tier  issue  within  the  top  tier 
S  Middle  tier  issue  reported  in  IDA  and  RAND  RCAs 


Program  Count  by  Issue* 


Issues  at  Inception 
Flawed  Milestone  II  Cost  Estimate 
Lack  of  Acquisition  Rigor 
Immature  Technology 
Unrealistic  Performance  Expectation 
Issues  During  Execution 
Inadequate  Government  Management 
Unanticipated  Design  Issues 
Evolving  Requirements  and  Scope 
Poor  Contractor  Performance 
Fragmented  and  Diffused  Management  Structure 
Exogenous  Factors 
Directed  Changes/Priorities/Quantities 
Shrinking  of  Business  Base/Emerging  of  Low  Cost  Competitor 
Inadequate  Program  Funding 
Constricted  Environment  to  Control  Cost 


Number  of  Programs 


Figure  1.  Program  Count  with  Top  and  Middle  Tier  Issues 


Program  Count  by  Middle  and  Lower  Tier  Issues** 


Flawed  Milestone  II  Cost  Estimate 
Unrecognized  Technology  Requirement/Complexity 
Unrealistic  Schedule 
Unrealistic  Cost  Estimate 
Erroneous  Cost  Estimate 
Inadequate  Risk  Allowance 
Flawed  Programmatic  Assumptions 
Flawed  Weight  Estimate 
Directed  Changes/Priorities/Quantities 
Lack  of  Acquisition  Rigor 
Flawed  Contracting  Strategy/Structure/Incentive 
Adopted  Unrealistic  Cost  Estimate 
Flawed  Engineering  Strategy 
Ambitious  Requirement 
Immature  Design 
Inadequate  Government  Management 
Inadequate  Oversight/Situation  Awareness 
Ineffective  Management  of  Contractor  Incentives 
Poor  Schedule  Management 
Deficient  Management/Funding  Practice 
Unanticipated  Design  Issues 
Evolving  Requirements  and  Scope 
Immature  Technology 
Unrealistic  Performance  Expectation 
Poor  Contractor  Performance 
Scheduled  Delivery/Defect  Rate/Reliability 
Poor  Management  of  Subcontractor 
Fragmented  and  Diffused  Management  Structure 
Shrinking  of  Business  Base/Emerging  of  Low  Cost  Competitor 
Inadequate  Program  Funding 
Constricted  Environment  to  Control  Cost 


**  Blue  -  middle  tier  issue 
Purple  -  lower  tier  issue 


Number  of  Programs 


Figure  2.  Program  Count  with  Middle  and  Lower  Tier  Issues 

The  number  of  specific  root  causes  per  program  varies  substantially  across  the 
sample  shown  in  Table  4.  Navy  ERP  has  no  issues,  DDG  1000  and  Excalibur  have  two 
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each,  and  Global  Hawk  has  eight  root  causes  responsible  for  its  Nunn-McCurdy  breach. 
(See  Table  A-l  in  Appendix  A  for  the  assignment  of  lower-tier  root  causes  to  the  fifteen 
programs.)  Interestingly,  the  number  of  root  causes  per  program  does  not  appear  to  be 
related  to  the  degree  of  cost  growth  for  that  program  (Figure  3  and  Figure  4).  The  two 
programs  with  the  highest  unit  cost  growth — ATIRCM  and  Excalibur — have  four  or 
fewer  significant  root  causes  of  cost  growth  each,  and  the  program  with  the  greatest 
number  of  root  causes  (Global  Hawk)  has  the  least  PAUC  and  second  least  APUC  growth 
of  those  surveyed. 


Table  4.  Middle  Tier  Issue  Counts  by  Program 


GCSS-MC 

ECSS 

Navy  ERP 

ACWA 

ATIRCM 

CMWS 

Global 

Hawk 

F35  (JSF) 

RMS 

FAB-T 

Apache 

LB-Bklll 

DDG  1000 

WGS 

Excalibur 

JTRS 

GMR 

Issues  at 
Inception  * 

2 

1 

2 

3 

3 

2 

2 

1 

1 

3 

1 

1 

1 

3 

Issues  During 

Execution 

Issues* 

5 

1 

2 

1 

1 

4 

2 

1 

3 

1 

1 

Exogenous 

Factors* 

2 

2 

1 

1 

1 

1 

2 

1 

2 

Total  Issues 

7 

2 

0 

6 

4 

4 

8 

4 

3 

5 

5 

2 

3 

2 

6 

APUC  Cost 
Growth  %  ** 

39% 

282% 

32% 

23% 

57% 

55% 

7% 

31% 

25% 

27% 

159% 

45% 

PAUC  Cost 
Growth  %** 

43% 

291% 

25% 

14% 

57% 

80% 

25% 

26% 

86% 

18% 

211% 

136% 

Growth  over 
MSA 

Estimate 

641% 

73% 

28% 

Top  Tier  issue  reported  in  IDA  and  RAND  RCAs 


**  Average  Procurement  Unit  Cost  (APUC)  and  Program  Acquisition  Unit  Cost  (PAUC)  growths  measured 
against  program’s  current  baseline 
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APUCCost  Growth  &  Total  Issues 


Number  of  Issues 


Figure  3.  Issue  Count  by  Program  and  APUC  Cost  Growth 


PAUCCost  Growth  &  Total  Issues 


Number  of  Issues 


Figure  4.  Issue  Count  by  Program  and  PAUC  Cost  Growth 


A  flawed  MS  II  cost  estimate  is  the  most  prevalent  mid-tier  cause,  cited  for  twelve 
programs.  Unrealistic  schedule  and  unrecognized  technology  requirement/  complexity 
are  the  top  specific  root  causes  in  this  category.  The  more  detailed  Table  A-l  shows  a 
great  deal  of  overlap  between  these  two  causes  and  the  unanticipated  design  issue  (mid¬ 
tier)  during  execution:  seven  programs  have  traced  the  cost  growth  to  at  least  two  of  these 
causes.  These  three  issues  are  themselves  closely  related,  as  unrecognized  technology 
complexity  often  leads  to  schedule  delay  and  unanticipated  design  issues,  which  cause 
additional  delays. 
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Evolving  requirements  and  scope,  and  inadequate  government  management,  are 
mid-tier  root  causes  shared  by  many  programs.  Changes  in  program  requirements  and 
scope  implemented  through  numerous  contract  modifications  added  a  substantial  amount 
to  the  program  cost.  Inadequate  government  oversight  and  ineffective  contract 
management  failed  to  incentive  contractors  to  control  cost  and  produce  a  quality  product 
on  schedule.  These  are  managerial  issues  during  execution  with  opportunity  for 
correction  and  improvement.  They  deserve  further  investigation. 

Directed  quantity  reduction  is  the  most  cited  exogenous  cause  of  cost  growth.  Some 
programs  in  the  RCA  reports  have  their  procurement  quantity  reduced  by  half  or  more. 
Quantity  reduction  drives  up  PAUC,  as  a  program’s  acquisition  cost  is  amortized  over  a 
smaller  base. 

F.  Conclusions 

IDA  was  commissioned  to  locate  common  issues  in  and  remedies  for  the  root  causes 
of  cost  growth  across  fifteen  RCA  reports  for  MDAPs  and  MAIS  programs. 
Notwithstanding  the  limitations  described  in  Section  D,  five  specific  conclusions  can  be 
drawn  from  this  exercise.  First,  the  survey  shows  that  the  root  causes  of  a  program’s  cost 
growth  are  many.  The  reported  root  causes  vary  from  program  to  program  and  are  often 
interrelated.  Most  programs  have  more  than  one  significant  root  cause.  These 
observations  from  the  RCA  reports  stand  in  contrast  to  “single  cause”  theories,  which 
blame  extreme  cost  growth  across  the  MDAP  portfolio  primarily  upon  a  single  factor 
such  as  immature  technologies,  inadequate  systems  engineering,  or  acquisition  “culture” 
at  DoD.  Although  some  factors — like  schedule  underestimation,  quantity  reductions, 
requirements  changes,  and  ineffective  management — are  common  to  several  programs, 
none  are  present  in  more  than  two-thirds  of  the  RCA  reports.  Furthermore,  these  factors 
were  never  the  sole  source  of  cost  growth  identified  for  the  program  in  question.  Even 
conceding  the  limitations  of  the  RCA  reports  themselves,  it  is  clear  that  the  causes  of 
extreme  cost  growth  are  many,  not  one. 

Second,  from  the  prevalence  of  causes  of  cost  growth  at  program  inception  in  the 
RCA  reports,  we  can  conclude  that  the  rigor  of  the  MS  B  process  is  important  for 
controlling  extreme  cost  growth.  Fourteen  of  fifteen  RCA  programs  identified  issues  at 
inception  as  causes  of  cost  growth.  Unrealistic  cost  and  schedule  estimates,  and  flawed 
contract  and  contract  incentive  structure  have  been  cited  in  several  RCAs.  Once 
embedded,  issues  at  inception  are  difficult  to  overcome.  In  the  cases  analyzed,  programs 
were  often  directed  to  re-baseline  their  estimates  or  change  their  contract  structure  at 
program  restructure  to  contain  future  cost  growth.  A  more  rigorous  and  accountable  MS 
B  process  would  help  to  limit  the  incidence  of  birth  defects  that  lead  to  extreme  cost 
growth  in  MDAPs. 
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Third,  exogenous  factors  often  contribute  to  cases  of  extreme  cost  growth.  Nearly 
two-thirds  of  the  RCA  reports  (nine  of  fifteen  programs)  cited  exogenous  quantity 
reductions,  requirements  changes,  or  factors  in  the  acquisition  environment  as  responsible 
for  cost  growth.  Directed  quantity  reduction  drives  up  PAUC  as  a  program’s  acquisition 
cost  is  amortized  over  a  smaller  number  of  units.  Requirements  changes  and  scope 
expansions  lead  to  numerous  contract  modifications  and  increase  the  program  acquisition 
cost.  To  the  extent  that  these  factors  are  truly  exogenous  to  program  initiation  and 
execution,  their  influence  on  cost  growth  has  the  effect  of  exaggerating  the  severity  of 
MDAP  perfonnance  shortcomings  in  the  data.  Measures  to  control  requirements  growth 
and  acknowledge  the  costs  of  directed  changes  could  be  implemented  at  program 
restructure  to  manage  the  influence  of  such  exogenous  factors. 

Fourth,  the  RCA  reports  suggest  that  proper  management  of  MDAPs  is  important 
for  controlling  extreme  cost  growth.  Roughly  half  (eight  of  fifteen  programs)  of  the 
RCAs  reported  inadequate  government  oversight,  ineffective  contract  management,  or 
diffuse  management  structures  during  execution  as  significant  causes  of  cost  growth.  In 
some  cases  during  program  execution,  the  government  overlooked  reliability  and 
contractor  perfonnance  issues  and  pressed  on  with  additional  contract  awards  to  adhere  to 
an  unrealistic  schedule.  In  other  cases,  such  as  FAB-T  and  ECSS,  the  RCA  reported  that 
the  government  has  perfonned  poorly  because  the  program  office  was  understaffed, 
lacked  expertise,  and  the  program  manager  tenure  was  short. 

Causes  at  program  inception  and  exogenous  factors  can  be  unambiguously 
associated  with  given  amounts  of  cost  growth,  and  poor  management  and  oversight  issues 
act  to  exacerbate  the  consequences  of  cost  growth,  and  the  tie  is  difficult  to  quantify. 
Nevertheless,  these  execution  issues  are  even  more  important  to  be  specified,  because 
they  are  areas  for  improvement  and  remedial  actions. 

Finally,  this  exercise  highlights  the  need  for  improved  root  cause  taxonomy  beyond 
that  already  suggested  by  WSARA  and  PARCA.  Although  this  report  proposes  an 
alternative  taxonomy,  it  does  so  by  allowing  the  RCA  reports,  which  used  WSARA  as  a 
guide,  to  largely  determine  the  structure  beneath  PARCA’ s  top-tier  allocation  of  issues  at 
inception,  issues  during  execution,  and  exogenous  factors.  To  improve  the  consistency, 
understanding,  and  usefulness  of  cross-program  root  cause  analyses,  more  rigor  is  needed 
to  structure  future  root  cause  taxonomies.  A  broader  selection  of  programs  and 
participants  would  serve  well  in  future  efforts  towards  this  end.  Along  with  improving  the 
rigor  of  the  MS  B  process,  untangling  the  role  of  management  perfonnance,  and 
acknowledging  the  influence  of  exogenous  factors,  the  development  of  a  cross-program 
RCA  structure  stands  as  a  key  opportunity  for  improvement. 
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Appendix  A. 

Detailed  Root  Cause  Analyses  Summary 


Table  A-l  is  a  tabulation  of  root  causes  of  program  cost  growth  reported  in  the 
fourteen  IDA  and  RAND  reports.  The  causes  are  grouped  by  issues  at  inception,  issues 
during  execution,  and  exogenous  factors.  Entries  based  on  IDA  reports  are  in  black,  and 
entries  from  RAND  reports  are  in  red.  Mid-tier  entries  are  marked  with  and  lower 
tier  entries  are  marked  with  “x”. 


A-l 


A-2 


Table  A-1.  Detailed  Root  Cause  Analyses  Summary 


Causes  of  Cost  and  Schedule  Growth* 

GCSS-MC 

ECSS 

Navy  ERP 

ACWA 

ATIRCM 

CMWS 

Global  Hawk 

F35  (JSF) 

RMS 

FAB-T 

Apache  LB-  1 

Bklll 

DDG  1000 

WGS 

Excalibur 

JTRSGMR 

Issues  at  Inception 

Unrealistic  Performance  Expectation 

✓ 

✓ 

✓ 

Immature  Technology 

✓ 

✓ 

✓ 

✓ 

Lack  of  Acquisition  Rigor 

✓ 

V 

✓ 

V 

✓ 

V' 

V 

✓ 

Flawed  Contracting  Strategy/Structure/Incentive 

X 

X 

X 

X 

Flawed  Engineering  Strategy 

X 

Immature  Design 

X 

Adopted  Unrealistic  Estimate 

X 

X 

Ambitious  Requirement 

X 

Flawed  Milestone  II  Cost  Estimate 

V 

v' 

✓ 

V 

V 

V 

✓ 

✓ 

✓ 

s 

V 

V 

Unrealistic  Schedule 

X 

X 

X 

X 

X 

X 

Unrecognized  Technology  Requirement/Complexity 

X 

X 

X 

X 

X 

X 

Erroneous  Cost  Estimate 

X 

X 

X 

Unrealistic  Cost  Estimate 

X 

X 

X 

X 

Flawed  Weight  Estimate 

X 

Flawed  Programmatic  Assumptions 

X 

Inadequate  Risk  Allowance 

X 

Issues  During  Execution 

Unanticipated  Design  Issues 

V 

✓ 

✓ 

✓ 

✓ 

Fragmented  and  Diffused  Management  Structure 

V 

✓ 

Evolving  Requirements  and  Scope 

V 

V 

✓ 

Poor  Contractor  Performance 

V 

■s 

V 

Scheduled  Delivery/Defect  Rate/Reliability 

X 

X 

Poor  Management  of  Subcontractor 

X 

Inadequate  Government  Management 

V 

•s 

✓ 

V 

✓ 

✓ 

Inadequate  Oversight/Situation  Awareness 

X 

X 

X 

X 

X 

Deficient  Management/Funding  Practice 

X 

Poor  Schedule  Management 

X 

Ineffective  Management  of  Contractor  Incentives 

X 

X 

Exogenous  Factors 

Directed  Changes/Priorities/Quantities 

✓ 

✓ 

✓ 

V 

V 

V 

✓ 

V 

✓ 

Constricted  Environment  to  Control  Cost 

✓ 

Shrinking  Business  Base/Emerging  of  Low  Cost  Competitor 

✓ 

✓ 

Inadequate  Program  Funding  J 

V 

APUC  Cost  Growth  % 

39% 

282% 

32% 

23% 

57% 

55% 

7% 

31% 

25% 

27% 

159% 

45% 

PAUC  Cost  Growth  % 

43% 

291% 

25% 

14% 

57% 

80% 

25% 

26% 

86% 

18% 

211% 

136% 

Growth  over  MS  A  Estimate 

641% 

73% 

28% 

✓  Middle-tier  issue 

IDA's  findings  in  black 


Lower-tier  issue 
RAND's  findings  in  red 
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Abbreviations 


ACWA 

APB 

APUC 

ATIRCM/CMWS 

CHEM  DEMIL-ACWA 

CTOL 

DoD 

ECSS 

EMD 

ERP 

FAB-T 

GCSS-MC 

Global  Hawk 

IDA 

JSF 

JTRS  GMR 
MAIS 
MDAP 
MS 

PARCA 

PAUC 

RCA 

RMS 

STOVL 

WGS 

WSARA 

u.s.c. 


Chemical  Demilitarization  -  Assembled  Chemical 

Weapon  Alternatives 

Acquisition  Program  Baseline 

Average  Procurement  Unit  Cost 

Advanced  Threat  Infrared  Counter  Measure/Common 

Missile  Warning  System 

Chemical  Demilitarization-Advanced  ChemicalWeapons 
Alternatives 

Conventional  Take-off  and  Landing 
Department  of  Defense 
Expeditionary  Combat  Support  System 
Engineering  and  Manufacturing  Development 
Enterprise  Resource  Planning 

Family  of  Advanced  Beyond  Line-of-Sight  Tenninals 
Global  Combat  Support  System  -  Marine  Corps 
RQ-4A/B  UAS  Global  Hawk 
Institute  for  Defense  Analyses 
Joint  Strike  Fighter 

Joint  Tactical  Radio  System  Ground  Mobile  Radar 
Major  Automated  Information  System 
Major  Defense  Acquisition  Program 
Milestone 

Perfonnance  Assessments  and  Root  Cause  Analyses 

Program  Acquisition  Unit  Cost 

Root  Cause  Analysis 

Remote  Mine-Hunting  System 

Short  Take-off  and  Vertical  Landing 

Wideband  Global  Satellite 

Weapon  Systems  Acquisition  Refonn  Act 

United  States  Code 


D-l 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of 
information,  including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188), 
1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  V A  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any 
penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

xx-08-2013  Final 


4.  TITLE  AND  SUBTITLE 

Root  Causes  of  Nunn-McCurdy  Breaches — A  Survey  of  PARCA  Root  Causes 
Analyses,  2010-201 1 


3.  DATES  COVERED  (From  -  To) 


5a.  CONTRACT  NUMBER 

DAS  W0 1 -04-C-0003 

5b.  GRANT  NUMBER 


6.  AUTHOR(S) 

Lo,  Tzee-Nan,  K. 
Diehl,  Richard,  P. 
Gould,  Brandon  R. 


5c.  PROGRAM  ELEMENT  NUMBER 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


AY-7-3342 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Institute  for  Defense  Analyses 
4850  Mark  Center  Drive 
Alexandria,  VA  22311-1882 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


IDA  Paper  P-49 11 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Deputy  Director  for  Root  Cause  Analyses 

Office  of  Performance  Assessments  and  Root  Cause  Analyses 

Room  3C889A 

3620  Defense  Pentagon 

Washington,  DC  20301-3620 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

PARCA 


11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


14.  ABSTRACT 

The  Weapon  Systems  Acquisition  Reform  Act  of  2009  (WSARA)  requires  root  cause  analyses  (RCA)  be  conducted  on  major  defense  acquisition 
programs  (MDAPs)  with  extreme  cost  growth  for  its  Nunn-McCurdy  breach  certification  process.  In  2010  and  2011.  the  Director  for  Performance 
Assessments  and  Root  Cause  Analyses  (PARCA)  conducted  fourteen  RCAs  with  support  from  the  Institute  for  Defense  Analyses  (IDA),  the 
RAND  Corporation,  and  the  University  of  Tennessee.  IDA  was  tasked  to  identify  common  causes  exposed  by  these  RCAs.  IDA  developed  a 
taxonomy  to  classify  the  root  causes  reported  by  IDA  and  RAND,  compiled  a  survey  table,  and  identified  unrealistic  schedule,  evolving 
requirements,  inadequate  government  management,  and  exogenous  quantity  changes  as  the  most  frequently  cited  root  causes.  The  survey  shows 
that  the  root  causes  of  program  cost  growth  are  many,  not  just  a  “single  factor”  such  as  immature  technology  or  lack  of  systems  engineering.  The 
prevalence  of  causes  at  program  inception  shows  that  a  rigorous  Milestone  B  process  is  important  for  controlling  extreme  cost  growth.  Finally,  this 
report  concludes  that  management  of  MDAPs  matters,  as  almost  half  of  the  RCAs  reported  inadequate  government  management  as  a  significant 
root  cause  for  cost  growth. 


15.  SUBJECT  TERMS 

Nunn-McCurdy  Breach,  Root  Cause  Analyses,  Program  Acquisition  Unit  Cost  (PAUC),  Average  Procurement  Unit  Cost  (APUC), 
Cost  Growth,  Schedule  Delay,  Requirements,  Quantity,  Government  Management 


16.  SECURITY  CLASSIFICATION  OF: 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

Unclassified 

Unclassified 

Unclassified 

17.  LIMITATION  OF  18.  NUMBER  19a.  NAME  OF  RESPONSIBLE  PERSON 


ABSTRACT 


OF 

PAGES 


Husband,  D.  Mark 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

(571)256-1686 


Standard  Form  298  (Rev.  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


